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Autoregressive  process,  parameter  estimation,  CR  bounds,  linear  prediction, 
Prony's  method,  low-rank  approximation,  zero  selection,  modal  zeros, 
noise  zeros. 

In  this  paper  we  present  Cramer-Rao  (C-R)  bounds  for  parameter  estimation 
of  a  first-order  autoregressive  (AR)  process  from  a  finite  record  of  data. 

We  use  these  bounds  to  evaluate  the  performance  of  Maximum  Likelihood 
Estimation  (MLE)  and  linear  prediction  approaches.  Some  estimates  use 
low-rank  approximation  of  an  estimated  covariance  matrix.  The  latter 
estimates  are  based  on  the  method  of  Tufts  and  Kumaresan  [15].  Here  we 
have  added  a  zero  selection  technique  in  the  last  step  of  the  procedure. 
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Our  low-rank,  high  order,  linear  prediction  estimator  performs  better 
than  the  other  methods  which  we  have  tested,  when  the  pole  is  close  to 
the  unit  circle.  It  is  slightly  biased  and  its  variance  is  small  and 
close  to  the  variance  given  by  the  C-R  bound  for  unbiased  estimators. 
For  a  small  number  of  samples  (25  to  100)  this  estimator  performs 
substantially  better  than  the  MLE. 
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\  ABSTRACT 

In“ihis  paper^we^ presentrACramer-Rao  (C-R)  bounds  for  parameter 
estimation  of  a  first-order  autoregressive  (AR)  process  from  a  finite 
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record  of  data,  tfa  use!  these  bounds  to  evaluate  the  perfomance  of 
Maximum  Likelihood  Estimation  (MLE)  and  linear  prediction  approaches. 
Some  estimators  use  low-rank  approximation  of  am  estimated  covariance 

matrix.  The  latter  estimates  are  based  on  the  method  of  Tufts  and 
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Kumaresan  |J5J.  Here  s»e  have  added  a  zero  selection  technique  in  the 
last  step  of  the  procedure^ 
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•Over- low- rank,  high  order,  linear  prediction  estimator  performs 
better  than  the  other  methods  which  we  have  tested,  when  the  pole  is 
close  to  the  unit  circle.  It  is  slightly  biased  and  its  -variance  is 
small  and  close  to  the  variance  given  by  the  C-R  bound  for  unbiased 
estimators.  For  a  small  number  of  samples  (25  to  100)  this  estimator 
performs  substantially  better  than  the  MLE. 


*  This  work  was  supported  by  the  Probability  and  Statistics  Program 
of  the  Office  of  Naval  Research. 
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I.  INTRODUCTION 

For  the  past  seven  years  we  and  our  collaborators  have  been  working 
on  problems  of  parametric  signal  modeling.  We  have  tried  to  find 
estimators  of  signal  parameters  which  have  the  following  properties  : 
(1)  they  are  unbiased  or  their  biases  are  small  compared  with  the 
standard  deviation  of  error,  and  (2)  their  variances  are  close  to  the 
C-R  lower-bound  from  high  SNR  to  an  SNR  threshold  that  is  as  low  a3 
possible. 

For  the  purpose  of  estimation  of  signal  parameters  we  have 
introduced  the  use  of  low-rank*  approximation  to  data  matrices  or  to 
estimated  correlation  or  covariance  matrices  [l,9,10].  This  technique 
has  been  applied  in  parameter  estimation  for  sinusoids  [2], 
exponentially  damped  sinusoids  [5*4],  and  impulse  response  pole-zero 
identification  [4],  and  also  for  direction  finding  and  frequency  finding 
using  arrays  of  sensors  [6,8],  Simplified  calculations  have  also  been 
studied  [5.6,7]* 

During  the  last  few  years  we  have  turned  our  attention  to  the 
problem  of  parameter  estimation  of  random  signals  in  the  presence  of 
noise  for  short  records  of  data.  Both  autoregressive  (AR)  and 
autoregressive  moving  average  (ARMA)  models  have  been  used  [11-15].  We 
have  benefited  from  earlier  papers  on  system  identification  by  Mehra 
[l6],  Astrom  [ 1 7 ] ,  and  Parzen  [ 1 S] . 

The  modal  estimation  approach  [11,12],  which  starts  from  the 
estimated  correlation  coefficients  has  been  extended  [15]  to  include  the 
use  of  extra  modes  (poles)  for  modeling  the  effects  of  noise  and 


fluctuations.  This  leads  to  greater  accuracy  in  estimating  the  signal 
modes.  The  resulting  spurious  modes  can  be  suppressed  using  low  rank 
approximation  [15]  or  by  subset  selection  techniques  [5]. 

In  this  paper  we  more  thoroughly  evaluate  the  performance  of  the 
reduced-rank  modal  parameter  estimator  [15]  for  a  first-order  AR 
process. 

We  use  a  special  procedure  for  selection  of  the  signal  zero  from 
the  set  of  zeros  of  the  resulting  prediction-error-filter  polynomial. 

The  reduced-rank  approximation  can  be  carried  out  on  the  data 
matrix  of  the  linear  prediction  equations,  or  equivalently  on  the 
estimated  covariance  matrix  obtained  from  premultiplication  by  the 
transpose  of  this  data  matrix. 
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II. ANGLES  BETWEEN  TRUE  AND  ESTIMATED  EIGENVECTORS 

Recently,  through  perturbation  analyses,  which  were  motivated  by 
the  work  of  Wilkinson  [l9]  and  which  have  their  roots  in  the  work  of 
Rayleigh,  Timoshenko,  and  Courant,  D.  Tufts  has  traced  the  improvements 
of  low  rank  approximation  to  the  following  fact: 

The  angles  between  the  principal  eigenvectors  of  a  true  covariance 
matrix  and  the  corresponding  principal  eigenvectors  of  an  estimated 
covariance  tend  to  be  very  small,  especially  if  the  modal  pole  locations 
are  close  to  the  unit  circle.  We  now  present  the  results  of  measurements 
of  some  of  these  angles  performed  by  I.  Xirsteins  using  simulation  of  a 
first-order  AR  process. 

The  j'th  signal  vector  s.  generated  by  the  j'th  independent 
realisation  of  a  first-order  AR  process,  is  described  by  the  formulas 

T 

sj  -  L  s(1 ,4)  3(2, j)  ...  s(N,j)  ]  (1) 

s( t, j)*  a  s( t— 1 ,  j )  +  w(t,j)  ;  t*1,2,...,N  (2) 

i  ,2, .. . ,K 

where  {w(t,j)}  and  ls(0,j)}  are  mutually  independent  Gaussian  random 
variables  with  sero  mean  and  variances  <r*  and  »*/(l-a*)  respectively. 
Such  vectors  are  segments  of  independent  realizations  (index  j)  of  the 
AR  process.  The  signal  covariance  matrix  is  estimated  by 


X 


j-1 


In  our  experiments  we  lat  a  “.9  ,  H*10  and  K»1 6  or  K*32. 


The  true  correlation  matrix,  given  by  the  expected  value  of  (3), 
has  the  following  eigen-decomposition  : 


S 

a  -  s[  a  ]  -  v  a  vT-  v  vj 

i*i 


(4) 


where  V  contains  the  orthonormal  eigenvectors  [  V; }  of  R  and  A  is  a 
diagonal  matrix  containing  the  corresponding  eigenvalues  i  A  }.  We 

A  A 

define  the  i'th  eigenvectors  of  R  and  R  as  v,  and  V;  ,  respectively. 

The  angle  in  degrees  between  these  two  i' th  eigenvectors  is  given 
by 

130  .  T  . 

<t>  *  — . .  cos  |  j  y  |  (5) 

i  IT  it 


Histograms  of  the  absolute  values  of  4,  and  <t>z  for  500  simulation 
trials  are  presented  in  Figures  la  to  Id.  Two  values  of  K  ,  the  number 
of  independent  vector  observations  of  the  true  AR  sequence,  are  used. 
The  results  illustrate  the  angular  stability  of  the  estimated  principal 
eigenvector  v,  when  the  value  of  (a)  is  close  to  unity. 


In  the  context  of  autoregressive  processes  ,  as  in  the  case  of 
deterministic  signals  [l,10],  we  need  C-R  bounds  for  the  variance  of 
unbiased  estimates  of  the  parameters  of  interest,  in  order  to  have  a 
standard  of  comparison  for  existing  and  proposed  estimates.  Until  our 
work  such  bounds  appear  to  have  been  available  only  for  the  asymptotic 
case  of  very  long  observations  [20-23].  Here,  we  are  interested  in  cases 
for  which  the  observations  are  of  short  duration. 

Suppose,  as  described  above,  that  we  observe  iC  independent 
realizations  of  a  set  of  N  samples  of  a  steady-state,  first-order  AR 
sequence.  The  j'th  realization  {s(t,j)|  for  t*1,2,...,N  is  used  to  form 
the  j'th  row  of  a  matrix  Y  of  data.  We  start  with  the 
joint-multivariate-normal-density  function,  named  (f)  ,  of  the  K-rows  of 
Y,  conditioned  on  the  2-parameter  vector  @  : 

§  *  L  t  9Z] T  (6) 

in  which  0(  is  the  first-order  AR  parameter,  denoted  by  (a)  in  formula 
(2),  and  02  i3  9  *  the  standard  deviation  of  the  white-noise,  (w(t,j)}, 
of  (2). 


We  can  then  derive  the  elements  of  the  Fisher  Information  Matrix 
associated  with  this  process  by  the  following  steps  [24]: 

,2 


Ju  “  S  [  -  —  ln(  f)  ] 

J  MM 

•  j 


(7) 


-K  3  2  d 

—  — — [ln(  1  -  0|)]  ♦  kN  «— »[ln(  02)] 

2  MM. 

I  I  r 


X  -I 

♦  - tr[R  —(H)] 

2  MM 

•  j 


(3) 


2 


.  V.  -  . 


After  evaluation  of  these  derivatives  for  ia1 ,2  and  j-1 ,2  we  can 
express  the  fisher  Information  Matrix  in  the  following  form: 


1  -  2 )  ( 1  - 
(  1  -  a2)2 


<r  (  1  -  a  ) 


J  -  X 


<r(  1  -  a2) 


The  bounds  for  the  variance  of  the  estimated  parameters  are  then 
determined  from  the  diagonal  of  the  inverse  of  J  . 


var(a)  } 


(  1  -a2)2 

K[l  ♦  a2*  (N  -  2)(1  -  a2)] 

N(1  -  a2)2 

XN[l  ♦  aZ ♦  (S  -  2) ( 1  -  a)]  -  2Ka* 


;  <j  known 


;  a  unknown 


var(ff) 


[l  ♦  a2*  (S  -  2)(1  -  a2)] 


2KN[l  ♦  a  ♦  (N  -  2)(1  -  a  )]  -  4Ka 


;  a  known 


■2  ;  a  unknown 
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We  show  in  Figures  (2a-2h)  some  curves  of  the  above  C-R  bounds  for 
var(a) . 

The  asymptotic  bound  [20]  can  of  course  be  misleading  for 
small-sample-size  observations.  We  have  added  this  bound  in  Figs.  (2c) 
and  (2d)  only  for  a  reference.  One  can  verify  from  Fig. (2c)  that  as  (a) 
tends  to  unity,  the  asymptotic  bound  is  much  larger  than  the  exact 
bound.  Fig.  (2d)  presents  a  case  vhere  the  asymptotic  bound  is  lower 
than  the  true  ones  (for  a  known  and  unknown).  Also  it  is  interesting  to 
note  that  as  (a)  tends  to  zero  the  asymptotic  and  the  true  bounds 
converge  to  1 /N-1  . 

Figs.  (2a, 2b)  show  the  dependence  of  the  C-R  bounds  on  (a).  They 
can  help  one  to  visualize  a  continuous  transition  between  the 
decorrelated  (  a-O)  and  the  highly  correlated  case  (  a  — ►  ?  ). 

In  Figs  (2e,2f)  and  (2g,2h)  we  show  the  bounds  versus  the  number  K 
of  independent  realizations  for  two  different  values  of  (a)  for 
a  known  and  unknown. 
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IV.  THE  MAXIMUM  LIKELIHOOD  ESTIMATOR 

For  one  realization  (X*1 )  and  N  data  samples  of  a  first  order  A R 
sequence,  the  log- likelihood  function  is: 


1  2  1  2 
L-  - ln(1  -  a  )  -  Sln(  a) - -  (S00-  2a  S0I  ♦  a‘s„)  (12) 

2  2<rZ 


where : 


N 


8-1 


'oo 


S3*2 !  3»'S 


3t  3 1.|  S  O,, 


t-1 


t-1 


8-1 


z 


t*2 


3t2  (13) 


Therefore,  the  value  of  (a)  which  maximizes  (12)  ,i.e.  the  Maximum 
Likelihood  Estimator  of  (a)  ,is  a  solution  of  the  third  order  equation  : 


S01(H-2)  (S00*  S  S,i  )  8  S0, 

c  -  i i-i  cz  -  ■  ■  ■■  c  ♦  —  ■  —  ■  -  0  (14) 

S„  (S-l)  S„(B-1)  S„(H-l) 


The  maximum  likelihood  estimate  is  the  real  solution  of  (14)  of 
modulus  less  than  the  unity,  which  maximizes  (12). 

In  Figs.  3a-3c,  4a-4c,  we  show  the  curves  of  estimated  rms  error, 
variance  and  bias  of  the  MLE  of  (a)  which  have  been  obtained  by 
simulations  using  500  independent  realizations.  Comparison  with  the  C-3 
bound  curves  in  those  figures  3hov  that  for  small  sample  sizes  and 
values  of  (a)  near  unity,  one  might  be  able  to  obtain  better  performance 
than  that  of  the  MLE.  In  the  next  section  we  present  a  method  for  doing 


thi3 


V.  AH  SVD-BASBD  ESTIMATOR  FOR  THE  A-R  COEFFICIENT  0?  A  FIRST -ORDBR 

A-R  PROCESS 

For  the  special  problem  of  estimating  parameters  of  damped  or 
undamped  sinusoids  in  noise,  some  techniques  based  on  low- rank 
approximation  via  Singular  Yalue  Decomposition  (SYD)  have  been  shown  to 
possess  nearly  optimal  properties  [2], [4].  That  is,  the  standard 
deviation  of  the  estimation  error,  <ra  ,  is  very  close  to  the  value  given 
by  the  OR  bound.  And  the  bias  is  small  compared  with  the  value  of  <j%  . 

One  of  the  conditions  for  good  performance  of  these  techniques  is  a 
rank  deficiency  of  the  signal-alone  data  matrix  or  correlation  matrix. 

Processes  other  than  those  of  deterministic  type  can  possess  a 
near- to- rank-deficiency  of  the  correlation  matrix.  That  is,  after  a  few 
large  eigenvalues,  the  eigenvalue  spectrum  rapidly  falls  to  low  values 
(see  Figs.  5a, 5b, 5c).  We  can  then  define  subsets  of  random  processes, 
possessing  only  a  few  high-variance  components  in  their  Karhunen-Loeve 
expansions  over  finite  data  intervals. 

As  an  example  of  such  a  process,  let  us  consider  a  discrete, 
first-order  AR  process,  the  pole  of  which  is  located  on  the  positive 
real  line,  near  the  unit  circle. 

Let's  consider  the  eigenvector  outer-product  decomposition  of  the 
corresponding  (p  x  p)  true,  population  covariance  matrix  Rp  : 
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For  small  values  of  p,  a  good  approximant  of  Rp  is  the  compoaent 
carrying  the  most  of  the  variance  [27] . 

X,  y(  v(T  (16) 


As  the  size  p  of  the  covariance  matrix  becomes  larger,  more  terms 
are  required  for  a  given  level  of  approximation  (cf.  Figs.  5a, 5b, 5c). 

This  approximation  is  particularly  interesting  in  the  practical 
case  of  a  finite  observation  of  data  in  which  the  angle  of  the  estimated 
principal  eigenvector,  relative  to  the  true  principal  eigenvector  of  Rp, 
is  small.  That  is,  y,  and  Zi  are  nearly  identical  vectors. 

The  original  [28,29]  and  low-rank  linear  prediction  equations  can 
be  written  as  follows  : 


(original) 

Rp  g  -  -  r 

(17) 

( low-rank) 

(  X,v,  yJ)  $  -  -  r  . 

(13) 

If  we  represent  the  correlation  vector  r  using  the  eigenvectors  of 


Rp  : 


a  *  M  •••  * 


9  ”P 


(19) 


then  the  minimum-norm  solutions  for  the  coefficients  of  the  linear 
predictors  are  respectively  : 

p 


£ 


i  *  I 


li 


(20) 


b  -  -  (-M  v, 

X| 


(21  ) 


Using  the  prediction  coefficients  specified  by  (20)  and  (21 )  we  can 
form  the  associated  prediction-error-filter  polynomials.  The  set  of 
zeros  of  a  prediction-error-filter  polynomial  is  denoted  by  { z {  ; 
i*1,...,p  }.  The  positive  real  zero  nearest  to  the  unit  circle  is 

denoted  by  zq  . 

I  1  -  zq|  *  min  {  I  1  -  zj  ;  Zj  is  positive  real}  (22) 

i-1 ,...,p 


Having  found  zq  ,  if  it  exists,  we  use  it  to  estimate  the  A 3 
parameter. 

zq  ,  I  Zql <  1 

a  ■ 

1  / Zq  ,  1  Zq| >  1 

(23) 

If  zq  does  not  exist,  because  there  are  no  positive  real  zeros, 
then  we  make  no  estimate  of  (a).  In  practice,  this  could  certainly  be 
refined  by  projection  of  zeros  which  are  very  close  to  the  positive  real 
line. 


The  estimator  resulting  from  the  zero  selection  of  formulas  (22) 
and  (23),  when  applied  to  the  prediction-error- filter  resulting  from 
(20),  is  known  to  be  unbiased,  given  the  true  Kp.  On  the  other  hand  it 
may  be  slightly  biased  when  using  (21)  instead  of  (20).  This  can  be  seen 
for  two  values  of  (a)  in  Tables  |.l]  and  i.2]  and  in  Fig.  6,  where  the 
prediction-error- filter  zeros  resulting  from  (21 )  are  plotted  for 
different  predictor- lengths  (p).  The  presence  of  the  bias  is  not  a 
serious  drawback  in  the  estimator,  especially  when  the  data  record  is 


short,  and  the  poles  are  close  to  the  unit  circle,  in  which  case  the 
biases  may  be  lower  than  the  minimum  standard  deviation  given  by  the  C-R 
bound. 

In  Tables  [0  and  [2],  we  compare  the  biases  for  two  different 
values  of  (a):  .95  and  .99  .Vfe  cam  observe  that  the  bias  decreases  as 
(a)  tends  to  unity,  and  it  increases  as  the  predictor  length  gets 
larger. 

Thus,  as  one  would  expect,  if  the  true  covariance  matrix  is 
replaced  by  a  different,  rank-1  approximating  matrix,  some  error  is 
introduced  in  the  value  of  the  AS  parameter  which  is  obtained  using 
linear  prediction.  As  we  show  below,  the  acceptance  of  this  small  error 
with  exact  covariance  information  leads  to  improved  insensitivity  to  the 
errors  in  an  estimated  covariance  matrix. 


VI. SIMULATIONS 


In  this  ssctlon  we  present  some  results  of  simulations,  where  we 
compare  the  performance  of  maximum  likelihood  estimation  and  least 
squares  techniques  based  on  linear  prediction.  The  linear-prediction 
approach  is  here  carried  out  in  two  ways,  using  the  estimated  full- rank 
correlation  matrix  and  using  its  low-rank  approximant. 

Given  the  recursion  (2),  K*500  independent  realizations  were 

generated,  We  have  started  from  independent  initial  conditions  ls(0,j)  ; 

,500}  given  by  a  zero-mean  gaussian  random  number  generator  of 
variance  equal  to  0-/(1 -a  )  .This  was  done  in  order  to  guarantee  the 
stationarity  of  the  sequence  |s(t,j)  ;  t“1,...,N{  for  a  given 

trial  j  .  Without  loss  of  generality  <r  has  been  normalized  to  unity. 

We  have  used  the  Forward  (?)  and  the  Forward-Backward  (FB) 
[25], [26]  structures  for  the  data  matrix  I  defined  as  follows: 

?-data  matrix  ;  Y  -  I  ;  F-data  vector  :  h  *  h 

f  "f 

— 

r 

f 

FB-data  matrix  :  I  -  — — 

I 

b 


These  matrices  and  vectors  are  written  out  more  explicitly  as 
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”  s(p) 

s(p-1 ) 

...  3(2) 

3(1 )  “ 

s(p*1 )— 

s(p+1 ) 

s(p) 

...  3(3) 

3(2) 

s(p+2) 

Y  - 

e 

e 

e 

e 

h  - 

e 

f 

e 

e 

e 

e 

f 

e 

s(3-1 ) 

s(H-2) 

...  s(N-p+1 ) 

3(S-p) 

3(tt) 

”  s(2) 

3(3) 

...  s(p) 

3(p+1 )“ 

3(1  ) 

s(3) 

3(4) 

...  s(p+0 

s(p>2) 

3(2) 

y  - 

e 

e 

e 

e 

h  * 

e 

b 

e 

v 

e 

e 

b 

e 

s(3-p> 

1)  . 

....  s(S-1 ) 

s(s) 

a(N-p) 

(25) 

For  all  caaaa  we  have  taken  predictor* lengths,  p,  to  be  leas  than 
half  the  observed  data- vector  length  3. 

p  <  3/2  (26) 


Since  our  estimated  correlation  matrix  is  proportional  to  the 
transpose  product  of  the  data  matrix  Y  ,  namely  YTY,  the  theoretical 
formulas  for  the  prediction  coefficients,  (20)  and  (21),  can  be 
rewritten  for  the  available  data  as: 


g  *  -(YTY)*'YTh  -  -ftp  r  (27) 


A 


(23) 


As  opposed  to  the  case  of  (20)  and  (21),  the  scalars  and  vectors  of 
(27)  and  (28)  are  determined  by  the  observed  data  vector  of  N  samples. 

The  prediction  coefficients  of  (28)  have  been  previously  used  in 
[15]  where  the  objective  was  spectral  peak  estimation  of  an  AS  process 
observed  in  the  presence  of  extra  noise.  We  are  now  interested  in 
estimation  of  pole  location  for  the  first-order  AR  process  and  we  use 
the  zero  selection  technique  of  formulas  (22,23). 

Here  we  consider  the  case  where  only  one  realization  (K>1 )  of  a 
finite  record  of  data  (N)  is  available  for  estimation  of  the  AR 
parameter.  Five  hundred  independent  records  are  used  in  order  to  measure 
the  properties  of  the  estimation  error. 

The  simulation  results  shown  in  this  paper  have  been  obtained  from 
only  one  value  of  the  AR  parameter  ,i.e.  a  *.95  .  We  have  obtained 
similar  results  for  values  of  '(a)  varying  from  .8  up  to  .99  . 

The  noise- free  case  is  presented  in  Tables  3  through  14  from  data 
record  lengths  of  N*25  for  Tables  1  through  8  ,  and  11*100  for  Tables  9 
through  14.  These  results  are  summarized  in  Figs. 21 ,22  . 

The  signal-plus-noise  case  is  presented  in  Figs. 23, 24  for  an  SNR  of 
5db,  and  in  Fig. 25  for  an  SNR  of  Odb  (see  also  Tables  15  and  16). 

Associated  with  each  table  is  a  corresponding  figure  showing 
superpositions  of  zeros.  The  indices  (a,b,c)  are,  respectively,  the 


cases  of: 


a)  superposition  of  the  zeros  of  the  estimated 
prediction-error-filter  (P-E-P)  polynomials  for  all  500  trials  ; 

b)  superposition  of  the  zeros  of  the  estiaated  P-E-?  polynomials 
which  have  at  least  one  positive  real  zero  ; 

c)  superposition  of  the  zeros  of  the  estiaated  P-E-F  polynomials 
which  have  no  positive  real  zeros. 

If,  for  a  given  predictor  length,  we  have  at  least  one  positive 
real  zero  for  each  of  the  500  sets  of  P-S-F  polynomials,  the  figures 
associated  with  the  indices  (b)  and  (c)  are  not  plotted. 

For  the  noise- free  case  we  have  used  predictor  lengths  (p)  from  1 
to  6.  For  the  signal- plus- noise  case,  (p)  has  been  chosen  up  to  16. 

In  order  to  compare  performance  we  have  tabulated  some  information 
other  than  the  standard  deviation  and  bias  of  the  estimates  : 

1 )  the  number  of  zeros  outside  the  unit  circle  of  the  500  estimated 
P-E-F  polynomials  :  column  "uns"  ; 

2)  the  number  of  trials  in  which  the  selected  positive  real  zeros 
were  outside  the  unit  circle  :  column  "out"  ; 

3)  the  number  of  trials  in  which  there  was  no  solution  i.e.  ,  there 
was  no  positive  real  zero  :  column  "Opz"  ; 

4)  the  number  of  trials  in  which  there  was  only  one  positive  real 


zero  :  column  N1 pz"  ;  and 


5)  the  number  of  trials  in  which  there  were  two  positive  real 
zeros  :  column  "2pz". 

The  number  of  cases  with  more  than  two  positive  real  zeros  can  of 
course  be  deduced  from  3 ) ,  4 )  and  3 ) • 

Tables  1  and  2  give  the  asymptotic  (large  number  of  samples)  bias 
of  the  rank-1  approximation  method.  These  values  have  been  obtained 
numerically  from  (17)  and  (18),  given  the  true  correlation  matrix.  From 
Tables  4  and  6  one  can  see  that  thi3  estimator  may  be  viewed  as 
practically  unbiased  for  a  small  number  of  samples  (N*25)  if  one 
considers  the  bias  relative  to  the  lower-bound  standard  deviation  (C-R 


VII. COMMENTS  OH  THE  RESULTS 


The  Cramer-Rao  hounds  place  lower  hounds  on  the  estimation  error 
variances  for  unbiased  estimators.  The  maximum  likelihood  estimator  may 
perform  hadly  with  respect  to  the  OR  hound,  especially  near  threshold 
and  for  a  small  number  of  samples,  N.  As  conventional  methods  of  linear 
prediction,  including  all  of  the  standard  variations,  are  approximations 
to  maximum  likelihood,  the  same  comments  hold  for  linear  prediction 
techniques. 

The  defects  in  maximum  likelihood  and  conventional  linear 
prediction  can  be  corrected  by  tailoring  the  data  with  SVD  to 
incorporate  structural  information  (such  as  low  or  approximate- low 
rank),  fitting  a  model  to  the  data  that  is  substantialy  higher  order 
than  the  model  is  known  to  be,  and  separating  modal  zeros  from  noise 
zeros  using  prior  information  as  a  fitting  rule. 

Our  summarizing  results  of  Figs.  21  and  22  substantiate  these 
claims.  Beyond  this,  it  is  shown  in  Figs.  25,  24  and  25  that  even  more 
impressive  results  are.  obtained  in  the  presence  of  additive  noise. 

We  conclude  with  the  following,  more  detailed  comments  : 

a)  Classical  true-order  (p*1 )  linear  prediction 

For  a  small  number  of  samples  (N-25)  we  can  see  from  Tables  5  and  7 
for  p*1  that  the  Forward-Backward  Linear  Prediction  (FBLP)  performs 
better  than  the  Forward  Linear  Prediction  (FLP).  This  is  true  not  only 
for  rma  error  but  also  for  the  location  of  zeros  inside  the  unit  circle. 
For  N«25  we  had  57  out  of  500  cases  in  which  zeros  were  outside  the  unit 


circle  for  the  FLP  method,  while  for  the  FBLP  all  the  Prediction-Error 
Filters  (P-E-F)  were  minimum  phase  ("stable")* 

nevertheless,  the  FLP  and  FBLP  methods  tend  to  have  the  same 
performances  as  N  gets  large.  We  can  see  for  N*100  in  Tables  11  and  13 
that  for  p*1 ,  the  performance  is  about  the  same. 

From  Tables  3  and  9  one  verifies  that  the  MLE  is  slightly  better 
than  the  classical  true-order  linear  prediction,  but  still  the  C-R  bound 
is  not  achieved. 


b)  Full- rank  larger-order  (p>1 )  linear  prediction 

For  this  case  where  p>1 ,  using  the  full- rank  estimated  covariance 
matrix  and  our  particular  zero  selection  procedure,  we  also  conclude 
that  for  small  N("25)  the  FBLP  method  is  better  than  the  FLP  method. 
This,  in  terms  of  location  of  the  zeros  of  the  estimated  F-E-F 
polynomials  inside  the  unit  circle  and  in  terms  of  the  rms  error  (for 
p<5)-  'For  N“100  the  performances  of  both  methods  are  practically  the 
same  for  p<4. 

On  the  other  hand  the  number  of  trials  in  which  there  were  no 
positive  real  zeros  (Opz)  was  slightly  bigger  for  FBLP  than  for  FLP  for 
smaller  samples  (N*25)  ,cf.  Tables  (3,7)  and  (11,13). 

As  far  as  the  comparison  with  the  true  order  (p*l)  case  is 
concerned,  we  can  see  that  for  Naa23  the  full-rank  high-order  predictors 
provide  better  estimates,  especially  for  the.  FBLP  case.  The  optimal 
predictor  length  was  p»3  for  both  FLP  and  FBLP. 


On  the  other  hand,  for  N*100,  the  true-order  predictor  (p*l) 
performed  better  than  those  for  p>1  . 

One  could  infer  that  for  a  small  number  of  samples,  fluctuations  of 
the  estimated  covariance  are  not  negligible,  making  it  useful  to  use 
extra  poles  to  model  and,  hence,  isolate  the  fluctuations.  That  is,  the 
fluctuations  are  filtered  by  modeling  them.  But,  for  large-enough  data 
records,  the  covariance  is  accurate  and  the  true  order  can  be  used. 

If  we  look  at  Fig. 21  ,  we  can  notice  that  the  full-rank  high-order 
predictors  may  provide  better  estimates  han  the  MLB  for  a  small  number 
of  samples  (N*25).  This  is  the  case  for  FBLP  with  p*3» 

c)The  reduced- rank  high-order  predictors 

For  this  case,  the  value  of  p  is  greater  than  one  and  a  low- rank 
approximation  to  the  estimated  covariance  matrix  is  used  to  obtain  the 
prediction  coefficients.  This  method  provided  more  accurate  estimates 
than  the  other  methods  which  we  examined. 

Comparing  Tables  4  and  6  for  N“25,  and  Tables  10  and  12  for  N~100, 
once  more  the  FBLP  method  is  shown  to  perform  better  than  the  FLP  method 
This  can  be  also  viewed  in  Figs. 21  and  22. 

tfe  summarize  the  performances  of  these  estimators  for  the 
noise-free  case  in  Figs.  21  and  22  and  for  the  signal-plus-noi3e  case  in 
Figs.  23,  24  and  25. 

For  the  signal-plus-noise  case,  in  order  to  filter  the  noise,  we 
had  to  use  a  higher  order  predictor. 


After  completion  of  this  report  we  discovered  the  recently 
published  paper  (ICASSP  -  march  1984)  by  S.  Shon  [30 ]  in  which  the  C-R 
bound  for  the  first-order  AR  parameter  is  also  presented.  In  the  last 
ASSP  Spectrum  Estimation  Workshop,  in  Tampa  (november  1983)  we  have  also 
presented  the  same  bound  [31 ]•  Our  proposed  low- rank  estimator  provides 
better  performance  than  the  estimators  discussed  by  Shon,  as  we  have 
shown  in  this  report.  The  C-R  bounds  for  the  estimation  of  AR 
coefficients  of  a  general  p-order  AR  process  can  also  be  determined 
exactly  (for  small  samples)  [32]. 
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Fig.  1)  Histograms  of  the  absolute  value  of  the  angular  perturbations 
of  the  principle  eigenvectors  of  R  . 
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Pig.  3a 

Comparison  of  the  Cramer-Rao  (CR)  lower  bound  standard 
deviation  for  unbiased  estimators  of  the  first-order  AR 
parameter  (a)  with  the  estimated  root-mean- square  error 
(rmae)  of  the  Maximum  Likelihood  Estimator  (MLE)  of  (a), 
for  a  sample  size  of  N*25  over  500  independent 
realizations. 

Values  of  (a)  :  .0,  .1,  .2,  .3,  .4,  .5,  .6,  .7,  .8, 
.9,  .92,  .95,  .99 
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Fig.  3b 

Comparison  of  the  Cramer-Rao  (CR)  lower  bound  standard 
deviation  for  unbiased  estimators  of  the  first-order  AR 
parameter  (a)  with  the  estimated  standard  deviation  (sigma) 
of  the  Maximum  Likelihood  Estimator  (MLE)  of  (a),  for  a 
sample  size  of  Na25  over  500  independent  realizations. 

Values  of  (a)  :  .0,  .1  ,  .2,  .3 ,  .4,  .5,  .6,  .7,  .8, 

.9,  .92,  .95,  .99 
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Fig.  3c 

Estimated  bias  of  the  Maximum  Likelihood  Estimator  (MLE)  of 
(a),  for  a  sample  size  of  N*25  over  300  independent 
realizations. 


Values  of  (a)  :  .0,  .1,  .2,  .3,  .4,  .5,  .6,  .7,  .8 
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Fig.  4a 

Comparison  of  the  Cramer-Rao  (CE)  lower  bound  standard 
deviation  for  unbiased  estimators  of  the  first-order  AR 
parameter  (a)  with  the  estimated  root-mean- square  error 
(rmse)  of  the  Maximum  Likelihood  Estimator  (MLE)  of  (a), 
for  a  sample  size  of  JMOQ  over  500  independent 
realizations. 

Values  of  (a)  :  .0,  .1.  .2,  .3.  .4.  .5,  .6,  .7.  .8, 


First  order  A-R  parapet* r  (  a  ) 


Fig.  4b 

Comparison  of  the  Crane r-Rao  (CR)  lower  bound  standard 
deviation  for  unbiased  estimators  of  the  first-order  AR 
parameter  (a)  with  the  estimated  standard  deviation  (sigma) 
of  the  Maximum  Likelihood  Estimator  (MLE)  of  (a),  for  a 
sample  size  of  N«100  over  500  independent  realizations. 

Values  of  (a)  :  .0,  .1,  .2,  .3,  .4,  .5,  .6,  .7,  .8, 

•9,  .95,  .97,  .99 


First  order  *-R  parameter  C  a  ) 


Fig.  4c 

Estimated  bias  of  the  Maximum  Likelihood  Estimator  (MLE)  of 
(a),  for  a  sample  size  of  2M00  over  500  independent 
realizations. 

V alues  of  (a)  ♦  •  Q ,  • 1 ^  •  2 ,  »3 1  .4*  •  5 ,  •  6 ,  *7 ^  .9. 

.9.  .95,  .97,  .99 


Eigenvalues  of  the  p-dimension  normalized  (trace-p)  correl 
matrix  Rp,  of  a  first-order  AR  process  in  which  a*. 95 


a  »  .95  ;  Noise-free  case  ;  N  *  « 

Zeros  of  the  Prediction-Error-Filter  Polynomial  for  the 
asymptotic  case  of  large  samples  from  the  rank-1  approximation 
of  the  (true)  correlation  matrix. 

[  p  :  predictor  length  j 


(cf.  Table  1) 


p»l  p*4  p«4;  n*496 


p*3  p»6  p*3;  n*499  p*6;  n*487 


Fig.  3a  Fig.  8b 

a  ■  .95  ;  Noise-free  case  ;  N  *  25 

Superposition  of  the  Zeros  of  the  Estimated  Prediction-Srror-Filter  Polynomials 
obtained  from  the  rank-1  approximation  of  the  estimated  Forward-Backward 
covariance  matrix  for  :  a)  All  500  trials.  b)  Cases  in  which  each 

set  of  polynomial  zeros  has  at  least  one  positive  real  zero. 

[  p  :  predictor  length  ;  n  :  number  of  cases  out  of  500  trials  j 


(cf.  Table  4) 


p«4;  n*4 


p*3;  nal  p-6;  na13 


Fig.  8c 

a  -  .95  ;  Noiae-free  ;  N  »  25 

Superposition  of  the  Zeros  of  the  Estimated  Prediction-Error-Filter  Polynomials 
obtained  from  the  rank-1  approximation  of  the  estimated  Forward-Backward 
covatiance  matrix.  c)Cases  in  which  each  set  of  polynomial  zeros  has  no  positive 
real  zero. 

[  p  :  predictor  length  ;  n  ;  number  of  cases  out  of  500  trials  ] 


(cf.  Table  4) 


p*2;  n* 462 


p*5;  n*378 


a  •  .95 


t 


Noise-free 


N  -  25 


Superposition  of  the  Zeros  of  the  Estimated  Prediction-Error-Filter  Polynomials 
obtained  from  the  estimated  full- rank  Forward-Backward  covariance 
matrix.  c)Cases  in  which  each  set  of  polynomial  zeros  has  no  positive  real  zero, 
p  :  predictor  length  ;  n  :  number  of  cases  out  of  500  trials  ] 


(cf.  Table  5) 


mm 
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p»3;  n*499 


p*6  ;  n=4aa 


Fig.  10a 


a  »  .95 


Fig.  10b 


Noise-free  case 


N  -  25 


Superposition  of  the  Zeros  of  the  Estimated  Prediction-Srror-Filter  Polynomials 
obtained  from  the  rank-1  approximation  of  the  estimated  Forward  covariance 


matrix  for  : 


a)  All  500  trials. 


b)  Cases  in  which  each  set  of 


polynomial  zeros  has  at  least  one  positive  real  zero. 

[  p  :  predictor  length  ;  n  :  number  of  cases  out  of  500  trials  ] 


(cf.  Table  6) 


p*3;  n«l  p»6;  n*12 


Fig.  10c 

a  ■  .95  ;  Noiae-free  case  ;  N  ■  25 

Superposition  of  the  Zeros  of  the  Estimated  Prediction-Error-Filter  Polynomials 
obtained  from  the  rank-1  approximation  of  the  estimated  Forward  covariance 
matrix.  c)Cases  in  which  each  set  of  polynomial  zeros  has  no  positive  real  zero. 
[  p  :  predictor  length  ;  n  :  number  of  cases  out  of  500  trials  ] 


(cf.  Table  6) 


a  -  .95 


f 


Noise-free  case 


N  -  25 


Superposition  of  the  Zeros  of  the  Estimated  Prediction-Error-Filter  Polynomials 

obtained  from  the  estimated  full-rank  Forward  covariance  matrix  for  : 

a)  All  500  trials.  b)  Cases  in  which  each  set  of  polynomial  zeros  has  at 

n  :  number  of  cases  out  of  500  trials  ] 


least  one  positive  real  zero. 
[  p  :  predictor  length 


(cf.  Table  7) 


p *2;  n*32 


p*5;  n*117 


Fig.  11c 

a  *  .95  ;  Noise-free  case  ;  N  *  25 

Superposition  of  the  Zeros  of  the  Estimated  Prediction-Error-Filter  Polynomials 
obtained  from  the  estimated  full- rank  Forward  covariance  matrix,  c)  Cases  in 
which  each  set  of  polynomial  zeros  has  no  positive  real  zero. 

[  p  :  predictor  length  ;  n  :  number  of  cases  out  of  5000  trials  ] 


(cf.  Table  7) 


p*2;  n«38  p*5;  n*21 9 


p*3;  n»l08  p=6;  n*251 

Fig.  12c 

a  *  .95  ;  Koiae-free  case  ;  If  »  25 

Superposition  of  the  Zeros  of  the  Estimated  Prediction-Error-Filter  Polynomials 
obtained  from  the  raaic-2  approximation  of  the  estimated  Forward-3ac!cward 
covariance  matrix.  c)Cases  in  which  each  set  of  polynomial  zeros  has  no  positive 
real  zero. 

L  p  :  predictor  length  ;  n  :  number  of  cases  out  of  500  trials  ] 


(cf.  Table  3) 


Fig.  13 


a  *  .95  ;  Noise-free  case  ;  N  -100 


Superposition,  of  tne  Zeros  33ti.nat9d  by  the  MLO  over  500 


trials 


(of.  Table  9) 


Fig.  14 


a  *  .95  ;  Noiae-free  case  ;  N  -  100 

Superposition  of  the  Zeros  of  the  Sstimated  Prediction-3rror-Filter  Polynomials 
obtained  from  the  rank-1  approximation  of  the  estimated  Forward-Backward 
covariance  matrix,  tfote  :  For  all  500  trials,  each  set  of  polynomial  zeros  has 
only  one  positive  real  zero. 

L  p  :  predictor  length  ] 


(cf.  Table  10) 


a. 


Pig.  15a  Pig.  15b 

a  *  .95  ;  Noise-free  case  ;  N  -100 

Superposition  of  the  Zeros  of  the  Sstimated  Prediction-3rror-?ilter  Polynomials 
obtained  from  the  estimated  full- rank  Forward-Backward  covariance  matrix  for  : 
a)  All  5000  triala.  b)  Cases  in  which  each  set  of  polynomial  zeros  has  at 

least  one  positive  real  zero. 

I,  p  :  predictor  length  ;  n  :  number  of  cases  out  of  500  trials  ] 


(cf.  Table  tl) 


a  *  .95  ;  Noise-free  case  ;  N  *100 

Superposition  of  the  Zeros  of  the  Sstimated  ?rediction-3rror-Filter  Polynomials 
obtained  from  the  estimated  full- rank  Forward- Backward  covariance 

matrix.  c)Cases  in  which  each  set  of  polynomial  zeros  has  no  positive  real  zero. 
L  p  :  predictor  length  ;  n  :  number  of  case  out  of  500  trials  ] 


(cf.  Table  1 1 ) 


a  -  .95 


Noise-free  case 


t 


N  »  100 


Superposition  of  the  Zeros  of  the  Estimated  Prediction-Error-Filter  Polynomials 
obtained  from  the  rank-1  approximation  of  the  estimated  Forward  covariance 
matrix.  Note  :  for  all  500  trials,  each  set  of  polynomial  zeros  has  only  one 
positive  real  zero. 

[  p  s  predictor  length  ] 


(cf.  Table  12) 


c 


p*4;  n*487 


p*5;  n=467 


p*6;  n*436 


Pig.  17a  Pig.  17b 

a  *  .95  ;  tfoise-free  case  ;  S  *  130 

Superposition  of  the  Zeros  of  the  Satiaated  ?rediction-2rror-?ilter  Polynomials 

obtained  froa  the  estimated  full-rank  Forward  covariance  aatrix  for  : 

a)  All  500  trials.  b)  Cases  in  which  each  set  of  polynomial  zeros  has  at 

n  :  number  of  cases  out  of  5 00  trials  ] 


least  one  positive  real  zero 
i  p  :  predictor  length 


(cf.  Table  13) 


p*5;  n*33 


pa3;  n=3 


p*6;  n*64 


Pig.  17c 

a  -  .95  ;  Noise-free  case  ;  N  -100 

Superposition  of  the  Zeros  of  the  Estimated  Prediction-Srror-Pilter  Polynomials 
obtained  from  the  estimated  full-rank  Forward  covariance  matrix,  c) Cases  in 
which  each  set  of  polynomial  zeros  has  no  positive  real  zero. 

L  p  :  predictor  length  ;  n  :  number  of  cases  out  of  500  trials  ] 


(cf.  Table  13) 


a  -  .95 


3oise-free  case 


N  -  100 


Superposition  of  the  Zeros  of  the  S3timated  ?rediction-Srror-?ilter  Polynomials 
obtained  from  the  rank-2  approximation  of  the  estimated  Forward-Backward 
covariance  matrix  for  :  a)  All  500  trials.  b)  Cases  in  which  each 
set  of  polynomial  zeros  has  at  least  one  positive  real  zero. 

L  p  s  predictor  length  ;  n  :  number  of  cases  out  of  500  trials  ] 


(cf.  fable  II) 


Fig  13c 


a  ■  .95  »  Noise-free  case  ;  N  »  100 

Superposition  of  the  Zeros  of  the  Estimated  Prediction-Srror-Filter  Polynomials 
obtained  from  the  rank-2  approximation  of  the  estimated  Forward-Backward 
covariance  matrix,  c)  Cases  in  which  each  set  of  polynomial  zeros  has  no 
positive  real  zero. 

L  p  :  predictor  length  ;  n  s  number  of  cases  out  of  500  trials  ] 


(cf.  Table  14) 


Pig.  19a 


Fig.  19c 
P*13  ;  q-92 


Fig.  19b 
p-13  ;  n*408 


a  -  .95 


3113  -  0  db 


S  -  100 


Superposition  of  the  Zeros  of  the  Estimated  Prediction-Srror-Filter  Polynomials 
obtained  from  the  estimated  full-rank  Forward-Backward  covariance  matrix  for  : 


a)  All  500  trials. 


b)  Cases  in  which  each  set  of  polynomial  zeros  has  at 


least  one  positive  real  zero.  c)Cases  in  which  each  set  of  polynomial  zeros 
has  no  oositive  real  zero. 


.  p  :  predictor  length 


n  :  number  of  oases  out  of  500  trials 


(cf.  Table  15) 


Fig.  20a 

Fig.  20b 

p-13 

p-13  ;  n-437 
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is  *  ^7 

jy 

Fig. 

,  20c 

p-13 

;  n-13 

a  -  .95 

;  SNR  - 

0  db 

3  *  100 

Superposition  of  the  Zeros  of  the  Sstimated  Prediction-Srror-Filter  Polynomials 
obtained  from  the  rank-1  approximation  of  the  estimated  Forward-Backward 
covariance  matrix  for  :  a)  All  500  trials.  b)  Cases  in  which  each  set  of 
polynomial  zeros  has  at  least  one  positive  real  zero.  c)Cases  in  which  each 
set  of  polynomial  zeros  has  no  positive  real  zero. 

[  p  s  predictor  length  ;  n  :  number  of  cases  out  of  500  trials  j 


(cf.  Table  16) 


predictor  length  (j>) 


Pig.  22 

Performance  (in  rmae)  of  estimators  of  a  first-order  AR 
parameter  (a*. 95)  from  segments  of  N-100  noise-free  samples 
of  data. 


predictor  length  (p> 


Pig.  24 

Performance  (in  rmse)  of  estimators  of  a  first-order  AR 
parameter  (a*. 95)  from  segments  of  N*100  samples  of  data 
corrupted  by  additive  white  noise,  such  that  the 
StHl-5db. 


predictor  length  <p> 


Fig.  25 

Performance  (in  rmse)  of  estimators  of  a  first-order  AR 
parameter  (a*. 95)  from  segments  of  N*100  samples  of  data 
corrupted  by  additive  white  noise,  such  that  the 
SNR-Odb. 


TABLS  t 


69 


•Ida*  ■  9.463  E-02 


■••a  •  0 .997 


bias  -  •5.313  1-02 


raa  •  1  .095  1-01 


TABLE  3 

*••95  s  Ioi*a-fraa  ;  J*25 
Iotas  C-l  (atjaa) •  4.931  S-02 

MAXIMUM  LIKELIHOOD  ESTIMATION 
(  of.  /i*.  7  ) 


p 

•  Ida* 

•••a 

bias 

raa 

1 

uaa 

out 

Op* 

1  p* 

2  pa 

1 

9.791 

E-02 

0.992 

-5.741 

E-02 

1 .135 

s-ot 

0 

0 

0 

500 

0 

2 

9.027 

3-02 

0.922 

-2.737 

B-02 

3.491 

E-02 

0 

0 

0 

500 

0 

3 

7.4IS 

E-02 

0.931 

-1 .933 

B-02 

7.640 

B-02 

0 

0 

1 

499 

0 

4 

7.906 

B-02 

0.936 

-1 .377 

B-02 

7.926 

B-02 

0 

0 

4 

496 

0 

5 

6.923 

S-02 

0.940 

-9.422 

B-03 

6.999 

2-02 

6 

6 

9 

492 

0 

6 

7.291 

3-02 

0.941 

-9.693 

2-03 

7.333 

3-02 

6 

6  • 

13 

497 

0- 

TABLE  4 

*•.95  1  lolaa-fraa  oaa*  ;  1*25 

£  Iota  1  5-8  ala  atsadard  davtatloa  (al«aa)  ■  4.331  3-02  ] 
Zaro  aalaotloa  aftar  raatc-1  approxlaatloa  oa  tba  aatlaatad 
/oraard-Baokvard  sovarlaaoa  aatrls 
(  of.  /id*.  3a,3b,3o  ) 


- 

1 

1  .35$  8-01 

0.888 

-6.111  8-02 

1 .202  3-01 

57 

57 

0 

900  0 

• 

2 

a. 940  8-02 

0.919 

-5.429  3-32 

9.976  8-32 

71 

71 

3 

900  0 

a. 277  8-02 


•2.757  S-02 


a.7ta  s-32  a;  a;  i  499 


7.962 

• 

o 

*> 

0.929 

-2.479  3-02 

3.558  8-02 

98 

98 

5 

497  0 

7.774 

3-02 

0.927 

-2.297  3-02 

8. 107  8-02 

137 

137 

a 

492  3 

a. 2i a  i-02 


-2.912  3-32  a. 994  3-02  119  119  12  484 


TA3L8  6 

•■.99  t  Jolao-frao  saaa  ;  3*29 
[  3o«a  i  3-8  ata  ataadard  dariatloa  (»i*aa)  ■  4.851  3-32  ] 
Zara  aolaotloa  aftar  raak-1  approximation  on  tba  aatiaatad 
forward  oowarlaaoa  aatrlx 
(  of.  fi«a.  I0a.l0b.13o  ) 


aigaa  •  3.652  B-02 


■••a  *  0.934 


bias  •  -1.615  B-02 


raa  -  3.993  l<02 


TAB  LS  9 

a*. 95  ;  Soiaa-fraa  ;  2*100 
Bota:  C-a  (ai«aa)*  2.382  3-02 

tUXIHOH  LI'iSLIHOOO  3371X12102 
(  af.  ?lf.  13  ) 


p 

alcaa 

aaaa 

blaa 

raa 

uaa 

out 

Opt 

*  p* 

2  pa 

1 

3.746 

B-02 

0.933 

-1 .680 

S-02 

4.106 

B-02 

0 

0 

0 

500 

0 

2 

2.567 

3-02 

0.955 

5.3*3 

B-03 

2.622 

B-02 

0 

0 

0 

500 

0 

3 

2.195 

B-02 

0.962 

1 .264 

8-02 

2.533 

B-02 

0 

0 

0 

500 

0 

4 

2.028 

B-02 

0.966 

1 .616 
% 

B-02 

2.594 

B-02 

0 

0 

0 

500 

0 

5 

1  .952 

B-02 

0.963 

1  .314 

B-02 

2.665 

B-02 

0 

0 

0 

500 

0 

6 

1  .917 

B-02 

0.969 

1  .941 

B-02 

2.729 

B-02 

0 

0 

0 

500 

0 

TiBLZ  10 

a*. 95  l  Soiaa-fraa  aaaa  ;  1*100 
L  Sota  s  C-3  ala  ataadard  dariatlan  (ai*«a)  ■  2.382  3-02  ] 
Zaro  aalaottoa  aftar  raak-1  approximation  oa  tba  aatiaatad 
Forvard-Baakvard  aorarianoa  matrix 


uas  out  Ops  t ps  2ps 


3.746  *-02 


4.096  3-02 


4.966  1-02 


9.302  3-02 


6.943  B-02 


t .004  3-01 


0.933 


0.932 


0.931 


0.929 


0.929 


0.929 


•  1.630  3-02  4.106  3-02 


•  1  .791  3-02  4.499  3-02 


-1  .399  3-02 


-2.079  3-02 


-2.024  3-02 


-2.464  3-02 


4.948  8-02 


6.162  3-02 


6.394  8-02 


1 .034  3-01 


260  240 


3  240  291 


224  246 


•tiaii  it 

S-.99  1  Solaa-fraa  oaaa  ;  3-100 

L  Sots  :  C-a  ala  standard  donation  (aigaa)  •  2.382  3-02  ] 
Zaro  salaotioa  from  tho  astiaatsd  full-rank 
forward- Backward  aovarlaaoa  aatrls 
(  of.  figs.  15a, 15b, 15c  ) 


3.771  3-02 


2.641  8-02 


2.330  3-02 


2.136  3-02 


2.117  3-02 


2.082  3-02 


0.933 


0.999 


0.962 


0.966 


0.963 


0.969 


-1  .691  3-02 


9.432  3-03 


1  .299  3-02 


1.997  8-02 


1 .789  3-02 


1.913  3-02 


4.117  3-02 


2.697  3-02 


2.647  3-02 


2.707  8-02 


2.772  3-02 


2.328  3-02 


9  9 


12  12 


13  13 


19  19 


0  500 


500 


7431.3  12 

a". 95  i  Soisa-froa  nasa  ;  S-100 
v,  Sota  i  C-B  ala  standard  donation  (slgaa)  •  2.332  3-02  ] 
Zaro  solaotioa  aftar  raak-t  approximation  an  tka  aatiaatad 
forward  aorarlaasa  aatrls 
(  of.  fig.  16  ) 


uns  out  Opt  I  pt  2  pa 


3.771  B-32 


4.137  3-32 


4.643  3-32 


3.371  3-02 


0.933 


0.932 


0.931 


0.929 


5  5.707  3-02  0.931 


.1.651  3-02 


■1 .726  3-32 


-1 .342  3-32 


•2.073  3-02 


-I  .339  3-02 


2.172  3-02 


4.117  3-02 


4.433  3-32 


4.999  3-32 


6.226  3-32 


5.996  3-02 


9.211  8-02 


261  239 


3  243  249 


2  2  13  226  246 


7A3LB  13 

a*. 95  s  Jotaa-fraa  oaao  ;  S-100 
:  C-i  ala  standard  day tattoo  (atgaa)  •  2.382  3-02  ] 
Zaro  salaotioa  from  tha  asttaatad  full-raax 
forward  oovartaaea  aatrtx 
(  of.  ft«s.  t7a.17b.17o  ) 


uaa  oat  Ops  1  pa  2 


! 

3.746  8-32  0.933  -1.680  3-02  4.106  3-02  3 

1  3 

3 

3  500 

4.396  3-32  3.932 


4.762  3-02 


3.940  3-02  0.929 


-1.751  3-02 


2.539  3-02 


2.797  3-02 


•2.320  3-02 


4.455  3-02 


5.42V  3-02 


5.496  3-02 


4.428  3-32 


3  263  2 


1  .534  3-02  3.327  3-02 


0  138  3  359 


6 


T43LB  14 

a*. 95  >  Jotaa-fraa  oasa  t  3-130 

i,  Sots  <  C-I  ala  standard  davtattoa  (atfaa)  •  2.382  3-32  ] 
Zaro  aalaottoa  aftar  raak-2  approataatloa  an  tbo  asttaatad 
forvard-Baokvard  oovartaaea  aatrtx 
(  of.  flea.  tSa.ISb.lSe  ) 


1  1.642  S-01  0.431  -5.191  E-01 

2  1.416  E-01  0.683  -2.666  E-01 

3  1.141  E-01  0.790  -1.596  E-01 

I  3.379  2-02  0.847  -1.030  E-01 

5  9.195  S-02  0.872  -7.752  S-02 

8.957  E-02  0.887  -6.265  B-02 

8.743  E-02  0.900  -4.965  E-02 

8  9.118  5-02  0.911  -3.894  E-02 

9  1.286  1-01  0.904  -4.555  1-02 

10  1.143  1-01  0.911  -3.870  1-02 

II  1.311  1-01  0.912  -3.760  1-02 

12  1.623  1-01  0.900  -4.969  1-02 

13  1.768  1-01  0.889  -6.147  1-02 

14  1.905  1-01  0.885  -6.462  1-02 

15  1.780  1-01  0.887  -6.337  1-02 

16  2.003  B-01  0.878  -7.207  E-02 


Tafcla  1 

••.95  !  5  •  100 

Z • ro  aalaotioa  from  th* 
Eorward-Baclcward  eov 


of.  Pigs.  19a, 19b 


1.641  E-01 


4.648  E-02 


4.682  E-02 


2  5.169  E-02 


3  3.812  E-02 


7.052  E-02 


5  5.315  E-02 


6  4.604  E-02 


5.191  E-01  5.444  E-01 


!  1.350 

E-01 

0.691 

-2.589 

E-01 

2.920 

E-01  l 

S  1.030 

E-01 

0.799 

-1 .506 

E-01 

1  .823 

E-01  I 

1  8.500 

E-02 

0.856 

-9.588 

E-02 

1  .281 

E-01  ( 

i  7.233 

E-02 

0.885 

-6.660 

E-02 

9.686 

E-02  ( 

i  6.405 

E-02 

0.906 

-6.385 

E-02 

7.762 

E-02  ( 

r  5.556 

E-02 

0.920 

-2.955 

E-02 

6.293 

B-02  ( 

5.601  E-02  0.928  -2.134  E-02  5.996  E-02 


5.399  E-02  0.936  -1.414  E-02  5.581  E-02 


7.789  E-03  4.713  E-02 


3.997  E-03  4.699  E-02 


2.802  E-03  5.177  E-02 


■953  3.640  E-03  3.829  E-02 


2.086  E-04  7.052  E-02 


953  3.273  E-03  5.325  B-02 


3.712  B-03  4.619  B-02 


tins  out  Ops  1  ps  2ps 


499 


500 


1  499 


500 


1  499 


500 


2  498 


500 


1  499 


496 


6  494 


5 


13  687 


8  492 


10  490 


20  480 


Tabla  16 

S-.95  ;  5  •  100  ;  SEE  -  0.  db 

Zsro  sslsctlon  aftar  ranic-t  approximation  on  tba  sstiaatad 
Forward-Backyard  covarianea  aatrix 

(cf.  Pigs.  20a,20bf20c  for  p*1J) 


